Efficient reduced state maximum likelihood sequence estimator

ABSTRACT

A reduced state maximum likelihood sequence estimator allows the use of improved equalization techniques that provides greatly improved performance for channels with severe attenuation and spectral nulls. The reduced state maximum likelihood sequence estimator retains k n  states of a total number of K states, k n &lt;K, with each retained state having an associated state metric. (J)(k n ) new states are determined based on k n  previous states and a most recently received sample, using J transitions, J being a less than L, where L is a size of a symbol alphabet. (J)(k n ) new state metrics are determined which are respectively associated with each new state. The new state metrics are compared to a threshold and those states whose metric does not exceed the threshold are retained. The reduced complexity of the MLSE allows for the use of partial response equalizers, e.g., a partial response class V (PRV) equalizer. The number of retained states k n  varies according to an adaptive threshold which limits the number of retained states between a lower bound and an upper bound. The threshold may be determined according to the metric of the most likely state of the retained states and a selected one of a plurality of transition metrics associated with the most likely state metric.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to data communications and in particular to anefficient reduced state maximum likelihood sequence estimator for use indata communication systems.

2. Description of the Related Art

The received signal in a digital communication system includes noise anddistortion caused at least in part by intersysmbol interference (ISI).One such digital communication system employs modems, which use digitalmodulation techniques to modulate and demodulate binary data over analogband-limited communications channels, e.g., telephone lines.

Modems typically conform to international standards to ensureinteroperability with modems from other manufacturers. One such standardis the V.34 specification described in ITU-T Recommendation V.34, AModem Operating at Data Signalling Rates of up to 28 800 bits/s for Useon the General Switched Telephone Network and on Leased Point-to-Point2-Wire Telephone-Type Circuits, dated September, 1994 (previously CCITTRecommendation V.34), which is hereby incorporated herein, in itsentirety, by reference. Another such standard is the V.90 specificationdescribed in ITU-T Recommendation V.90, A Digital Modem and AnalogueModem Pair For Use on the Public Switched Telephone Network (PSTN) AtData Signalling Rates of Up To 56 000 bits/s Downstream and Up To 33 600bits/s Upstream, dated September, 1998, which is hereby incorporatedherein, in its entirety, by reference.

Maximum Likelihood Sequence Estimators (MLSE) are widely used in datacommunications such as modem communications for decoding data passedthrough the channels with intersymbol interference (ISI) and fordecoding convolutionally encoded data. The Viterbi algorithm (VA) isquite often used to implement MLSE. A brief description of the Viterbialgorithm, which is well known in the art, is provided to facilitate anunderstanding of the present invention.

Assume that a sequence of symbols x={x_(i)}, i ε]−∝;∝[, x_(i)εA={a₁, a₂,. . . a_(L)} is transmitted through a channel with impulse responseh={h_(i)}, such that h_(i)≠0 only when 0<h_(i)<=M, where M−1 is thenumber of h_(i)story terms contained in the channel. The signal receivedat the other end of the channel in the absence of noise is defined asy=x*h, where “*” denotes a convolution. If the noise η is added to thesignal in the channel, then the signal at the other end of the channelcontains that noise and is defined as z=y+η. The receiver mustreconstruct the transmitted sequence of symbols x, based on the receivedsequence z. At the time nT the following can be used to reconstruct themost recently sent symbol x_(n):

x _(n)=(y _(n)−(x _(n−1) h ₁ +. . . +x _(n−M+1) h _(M−1)))/h _(o)  (1)

From the equation (1) it can be seen that besides the received signaly_(n), value x_(n) depends on the vector {x_(n−1), . . . x_(n−M+1)} ofM−1 previously decoded symbols. That vector will be further referred toherein as a state. With each new decoded symbol, the state of thedecoder changes to reflect the new value of the vector {x_(n−1), . . .x_(n−M+1)}. That is, if at a time nT the state was {x_(n−1), . . .x_(n−M+1)} then at a time (n+1)T the new state will be {x_(n), . . .x_(n−M+2)} and so forth. That change will be further referred herein asa state transition. The h_(i)story of state transitions represents astate trajectory. Since each symbol x_(n) belongs to a finite alphabet Aof size L, there can only be K=L^(M−1) unique states. All statescomprise a state set S={s₁, s₂, . . . s_(k)}, where each element S_(k)is a vector of M−1 elements that belong to alphabet A such thats_(k)≠s_(m) for k≠m.

Since the noiseless channel output y_(n) is not available to thereceiver, the value of x_(n) cannot be determined with absolutecertainty, and instead an estimate is used:

x _(n)=(z _(n)−(x _(n−1) h ₁ +. . . +x _(n−M+1) h _(M−1)))/h _(o)  (2)

The Euclidean distance, t_(n1), from the alphabet element a₁ to theestimate x_(n), is defined as t_(n1)−(x_(n)−a₁)², which provides themeasure of likelihood that the transmitted symbol x_(n) equals a₁ at thetime nT.

In a simple receiver, the alphabet element with the minimum Euclideandistance is selected and x_(n) is assumed to be equal to that alphabetelement. However if x_(n) is not determined correctly (e.g. due tonoise), that incorrect value will cause incorrect state transitioncausing future symbols to be decoded incorrectly affecting in turnfuture state transitions and so on. That condition is called “errorpropagation.”

To avoid the “error propagation” problem, all possible decoder statesand all possible state transitions must be examined. A state metricD_(n)={d_(n1), d_(n2), . . . , d_(nK)} defines the measure oflikelihood, such that each element d_(nk) defines the measure oflikelihood of state s_(k) at a time nT which is directly related to theEuclidean distance of the elements of the symbol vector corresponding tostate s_(k). The relationship of the state metric D and state set S isshown in FIG. 1 in which set S is shown to consist of four states s₁through s₄. In the example shown in FIG. 1 Aε(0,1) and L=2 and M=3. Eachstate s₁ through s₄ has an associated state metric d1 through d₄ in D.

1. For each state s_(k) of a set S calculate a symbol estimate x_(nk),based on received symbol z_(n) according to equation (2) above.

2. For each state s_(k) of a set S consider L transitions from thatstate, and for each transition combination (k, 1) calculate a combinedmetric d_(k1)=d_(k)+t_(k1), where t_(k1) is an Euclidean distance of thealphabet element a1 from the symbol estimate x_(nk),t_(k1)=(x_(nk)−a₁)².

3. For each state s_(k) of a set S consider L transitions to that state,selecting the one with the minimum combined metric d calculated duringstep 2 and assign its value to the state metric d_(k).

4. Find the state with the minimum metric d_(k) storing a transitionthat led to that state as a candidate optimal trajectory.

5. The first four steps are repeated for each value of n, i.e., for eachreceived symbol.

The optimal trajectory for the state with the smallest metric isconsidered the optimal path (most likely sequence) at a given time. Asit can be seen from the above description, the computational complexityof the Viterbi algorithm is proportional to L^(M). For high speedcommunication systems over band limited channels the value of L islarge, thus making the Viterbi algorithm complexity potentiallyprohibitive even for channels with relatively short impulse response.

One use of the Viterbi algorithm, as stated above, is in modem datatransmission. Such transmission is always affected by linear intersymbolinterference (ISI) and by noise. These two impairments are typicallycountered by equalization, a technique which minimizes ISI and noise atperiodic instances at which decisions are taken. The ultimate goal ofthe equalizer is to minimize a combined effect of ISI and noise and thusreduce the probability of incorrect decisions. Traditionally, dial upmodem designers used Linear Equalizers (LE) to do the job. Theperformance of LE is quite satisfactory when the combination of thechannel impulse response and the equalizer can form a Nyquist-1 pulse tosatisfy the distortionless (zero ISI) criterion. To achieve that goal,the bandwidth of the channel has to be wider than the symbol rateemployed by the modem.

The new generation of modems conforming to the V.90 recommendation nolonger satisfy this criterion. The impulse response of a PCM codec isdesigned to achieve satisfactory transmission of voice signals, and itsbandwidth is always more narrow than the 8000 Hz symbol rate used by amodem conforming to the V.90 standard. The spectral nulls at DC and 4000Hz generate infinite noise enhancements when LE is employed. To reducesuch noise enhancements, more advanced equalization techniques should beused. Those are decision feedback equalizers (DFE) and partial responseequalizers (PRE). Although usually treated separately, LE and PRE arejust special cases of DFE. DFE includes two portions: the forwardequalizer (FE) and the feedback equalizer (FBE). Both the FE and the FBEcan be implemented as linear FIR filters, where the input samples serveas an input to the FE and the previously decoded symbols serve as inputto the FBE. The output of both the FE and the FBE are added together andthe result is used as the basis for decision for determining thetransmitted signal. DFE allows some ISI to remain at the output of FE tobe then canceled by FBE. It has been shown that DFE can be the optimumreceiver with no decision delay. When, used in conjunction with Viterbidetector, (VD) DFE becomes a powerful tool in battling ISI even forchannels with spectral nulls.

However the implementation of a Viterbi detector grows considerably morecomplex with number of levels used by the transmission symbols and bynumber of taps in the FBE. The computational complexity of a Viterbidetector is proportional to L^(M) where M is the number of FBE taps andL is the size of the transmitted symbols alphabet.

Partial response equalizers (PRE), being a special case of DFE are usedto reduce the computational complexity of the Viterbi detector byforcing the FE to leave uncanceled only a controlled amount of ISI, thusrequiring only a small number of FBE taps to cancel it. The combinationof channel impulse response and FE of PRE form a response, calledPartial Response (PR), which contains only a few, usually integer-valuedcoefficients, thus allowing controlled levels of ISI from a small numberof symbols to be present at the instances of decision making. A Viterbidetector is then employed to remove that ISI and make the most probabledecision. The choice of partial response depends heavily on thetransmission channel. When spectral characteristics of the partialresponse match those of a channel, the FE has an easier task and noiseenhancement is limited. For a channel with spectral nulls, it is highlydesirable to use a partial response having spectral nulls at the samepositions to avoid infinite noise enhancement. A pulse code modulation(PCM) channel has two spectral nulls: one around DC, another around 4000Hz, thus confining the choice of the partial response for PCM modem tothose with the same spectral nulls. Class IV Partial Response (PRIV)often called modified duobinary, has long been used for the channelswith similar characteristics and is well known in the art.

The PRIV FE output contains the difference of two symbols spaced by twosymbol intervals:

y _(n) =x _(n) −x _(n−2)

Since ISI from only one symbol is present, the number of states of theViterbi detector equals L—the size of the symbol alphabet. For datarates achievable by a V.90 compliant modem, L<=128. Althoughimplementation of 128-state Viterbi decoder is not trivial, SetPartitioning, a well known technique in the art, can be used to reducethe number of Viterbi detector states to a more manageable number.However, when the channel has severe attenuation at a wider range offrequencies adjacent to DC and 4000 Hz, the noise enhancement by a classIV partial response equalizer becomes intolerable.

Therefore, it would be desirable to utilize a partial response equalizerthat does not exhibit intolerable noise enhancement and to have theperformance of a Viterbi decoder with manageable complexity.

SUMMARY OF THE INVENTION

Accordingly the invention provides a reduced state maximum likelihoodsequence estimator that causes only a slight degradation of performancerelative to an implementation of a full state Viterbi decoder. Thatallows the use of improved equalization techniques that greatly improveperformance for channels with severe attenuation and spectral nulls.

In one embodiment of the invention, a method is provided forimplementing a reduced state maximum likelihood sequence estimator in acommunications system to decode a received signal. The method includesretaining k_(n) states of a total number of K states, k_(n)<K, eachretained state having an associated state metric. (J)(k_(n)) new statesare determined based on k_(n) previous states and a most recentlyreceived symbol, using J transitions, J being a less than L, where L isa size of a symbol alphabet. (J)(k_(n)) new state metrics are determinedwhich are respectively associated with each new state. The new statemetrics are compared to a threshold and those states whose metric doesnot exceed the threshold are retained.

In another embodiment, the invention provides a modem that includes apartial response class V (PRV) equalizer coupled in its receive path. Instill another embodiment of the invention a modem includes a reducedstate maximum likelihood sequence estimator (MLSE) in its receive path.That MLSE may be coupled to a partial response equalizer. The reducedstate maximum likelihood sequence estimator retains a varying number ofreduced states. The number of retained states may be limited by use ofan adaptive threshold that limits the number of retained states betweena lower bound and an upper bound. The threshold may be determinedaccording to a metric of a most likely state of the retained states anda selected one of a plurality of associated transition metrics.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention may be better understood, and its numerousobjects, features, and advantages made apparent to those skilled in theart by referencing the accompanying drawings, wherein the use of thesame reference symbols in different drawings indicates similar oridentical items.

FIG. 1 illustrates the state vector and state metric utilized in Viterbidecoding.

FIG. 2 illustrates an exemplary modem incorporating an embodiment of thepresent invention.

FIG. 3 is a flow chart illustrating how reduced states and reducedtransitions are utilized in an embodiment of the present invention.

FIG. 4 is a flow chart illustrating the method of determining thethreshold according to one embodiment of the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

The current invention provides an approach which addresses the issue ofcomputational complexity of the Viterbi algorithm by offering anefficient approach which, while being suboptimal, causes only a slightdegradation of performance relative to an implementation of a full stateViterbi decoder. That allows the use of improved equalizationtechniques.

As discussed above, when the channel has severe attenuation at a widerrange of frequencies adjacent to DC and 4000 Hz, the noise enhancementcaused by the use of a PRIV equalizer becomes intolerable. One approachto battle this noise enhancement is to use Partial Response class V(PRV) equalizer. PRV FE output contains the second difference of symbolsspaced by two symbol intervals, and can be expressed as:

 y _(n) =x _(n)−2x _(n−2) +x _(n−4)

The PRV equalizer has double nulls at DC and 4000 Hz and thus producesmuch less noise enhancement for channels with high attenuation at bandedges. PRV equalizers were never considered practical for multilevelsignals such as those received in modems conforming to the V.90standard, because of the huge number of states required by Viterbidecoding. For example, a straightforward Viterbi implementation for adata rate of 56Kbits/sec having 128 symbols, requires 16,384 states.Such an implementation requires a significant amount of memory to storethe states as well as having high computational complexity because ofthe large number of states. Computational complexity of such Viterbidecoding is enormous even when set partitioning is employed. Plaindecision driven FBE cannot be used because of stability considerations.Even a single decision error will cause a subsequent chain reactionwithout any chance for recovery. The efficient reduced state MLSEdisclosed herein allows the use a wide variety of partial responseequalizers with performance close to that achieved by a full stateViterbi decoder without the enormous computational load required by afull state Viterbi decoder.

Another aspect of equalizer implementation is the tap spacing. T-spacedequalizers use the signal sampled at the symbol rate. Fractionallyspaced equalizers (FSE) operate on a signal sampled at a rate higherthan the symbol rate and require eventual down sampling. It is knownthat T-spaced equalizers depend heavily on coded sample clock phase, andthat there is at least one value of the phase in the symbol interval Tfor which the combined channel response and equalizer response havespectral nulls at half symbol rate. To avoid this undesirable effect,modem designers traditionally employed FSE in spite of its highercomputational complexity. However since PRV equalizers actually requirespectral null at half symbol rate, the advantage of FSE becomesnon-existent. Therefore a T-spaced equalizer is a logical choice for aV.90, modem. Use of T-spaced equalizer also allows the use of 8 KHzsample rate for the A/D converter.

A block diagram of an exemplary V.90 modem employing an embodiment of areduced state maximum likelihood sequence estimator and a class Vpartial response equalizer according to the present invention is shownin FIG. 2. A brief description of the modem shown in FIG. 2 will providethe appropriate context for better understanding the embodiments of theinvention described herein. A detailed discussion of the operation ofthe exemplary modem is not provided as modem operation in conformancewith, e.g., the V.90 recommendation, is known the art. In oneembodiment, the MLSE and partial response equalizer shown in FIG. 2 areimplemented in software which may be stored on computer readable mediawhich can be executed on a general purpose processor. Those of skill inthe art will appreciate that the teachings herein may also be applied toa variety of modem implementations such as those using a digital signalprocessors and/or stand alone processors.

The block diagram of the modem shown in FIG. 2 includes transmitter 201and receiver 203. The transmitter data module 205 supplies bits to theencoder 207. The transmitter module 205 either generates those bitslocally in the case of training or fetches them from the external datasource in the case of data mode. It then scrambles the bits as requiredand passes the data to the encoder 207. Encoder 207 converts the inputbit stream from data module 205 into a sequence of symbols which arethen provided to modulator 209. Modulator 209 performs appropriatesampling and filtering operations and provides its output to shaping andpreemphasis filter 211 that provides square-root-of raised cosineshaping as well as preemphasis filtering specified by the V.90recommendation. That output is provided to the digital to analogconverter 213.

The receiver portion of the modem 203 receives the output from thetransmitter (prior to the D/A) into receiver front end 215. Receiverfront end also receives the signal 216 from the channel. The front end215 performs such operations as analog to digital conversion on thereceived signal, various filtering and echo-canceling operations andprovides its output to the demodulator 217. The demodulator 217 includespartial response class V equalizer 219 as described further herein. Thedemodulator 217 also includes the second stage near end echo canceller220, which cancels the echo at the output of the equalizer and isintended to track slow variations of the echo signal. The output of thepartial response equalizer is provided to the decoder 221 and morespecifically to the reduced state maximum likelihood sequence estimator223, as described further herein.

Note that the noise signal at the output of PRV equalizer is stronglycolored due to the nature of the partial response used. Anoise-whitening filter 222 is required to achieve optimal performance ofthe MLSE. The noise-whitening filter is designed as an error predictor.It is trained during the reception of TRN signal during training phaseand then used in conjunction with the MLSE to minimize the errorprobability.

The symbols decoded by MLSE 223 are then split into sign and amplitudecomponents and passed respectively to the spectral de-shaper 225 andmodulus decoder 227. Both of those modules extract the data payload fromthe symbols that is later converted to a bit stream by data assembler229. Receive data module 231 descrambles this bit stream and passes itto the application program.

While a V.90 modem has been described, it should be understood thatother modems and other communication devices may advantageously use theinvention described herein.

Referring still to the block diagram shown in FIG. 2, the partialresponse equalizer 219 provides its output,(y_(n)=x_(n)−2x_(n−2)+x_(n−4)), to the reduced state MLSE 223. Becauseit is a reduced state estimator, at any given time, nT, the reducedstate MLSE described herein retains only a reduced number of statesk_(n). Note that k_(n) can vary, having a lower bound and an upper boundsuch that k₁<=k_(n)<=k₂. A further simplification as compared to thefull state Viterbi detector is that only a reduced number oftransitions, J, are analyzed, J<L, L being the size of the alphabet.

The advantages of the reduced state approach described herein arereadily apparent when compared with a full state Viterbi decoderimplementation. For example, for a full Viterbi decoder, 16,384 statescan be required for a modem conforming to the V.90 recommendation(alphabet of 128). In comparison, in one embodiment of the inventiondescribed herein, the reduced number of states may be only 12. Further,only two transitions (J=2) from those retained states are analyzedrather than L transitions (L being the size of the alphabet, e.g. 128).The parameters k₁ and k₂ are parameters of the reduced state MLSE whichcan be selected to adaptively control tradeoff between performance andcomputational complexity. In addition J can also be adjusted to increaseand/or reduce the complexity.

Referring to FIG. 3, the flowchart illustrates generally an embodimentof the reduced state estimator. The parameters k₁, k₂, k_(n) and J areinitialized. Some or all of the parameters may be implemented asprogrammable or may be fixed. Exemplary values are: k₁=8, k₂=16,k_(n)=12 and J=2. In 303, a noisy estimate of the received signal isdetermined for each of the retained states. Next, the MLSE determinesthe closest J alphabet elements to the noisy estimate in 305. In 307,the MLSE, using the J alphabet elements, determines the (J)(k_(n)) newstates. Thus, if there are 12 states in k_(n), and J=2, then 24 newstates will be generated. The transition metric associated with eachalphabet element is added to the state metric for each of the retainedstates to determine a state metric associated with each of the newstates. Finally, if the number of new states is outside the range of k₁and k₂, then the number of new states are adjusted accordingly.

If the number of new states is greater than k₂, the new states must bereduced down to the desired number of retained states. Otherwise, thenumber of retained states will grow, quickly causing the computationalcomplexity to be burdensome. Therefore, if (J)(k_(n)) (the number of newstates)>k₂, the MLSE discards those states with the largest metrics suchthat k₁<=the number of the remaining states<=k₂. In that way, the numberof retained states is manageable.

One approach to select the appropriate ones of the new states to retainis to sort the temporary state metrics in ascending order, and thenselect the first k_(n) new states to become the retained states. Howeversorting is an expensive operation with complexity of the order (k_(n)²). If k₁ and k₂ are large, sorting may become the dominant operation ofthe overall algorithm. An alternative to sorting is to determine anempirically based threshold “t.” The metrics of the new states arecompared to the threshold. Those new states whose metric is below thethreshold are retained. One way to calculate the threshold “t” is asfollows:

t=((k ₁ +k ₂)/(2Jk _(n))) (1/Jk _(n))Σd _(jn+1)

That formula is designed to provide a threshold that is adaptiveaccording to the number of retained states. Assume that k₁=8, k₂=16, J=2and the target k_(n) is 12. If the number of new states (J)(k_(n)) isexactly twice the target of 12, i.e. the number of retained states isexactly the target, then the first term of the equation above provides(½). That term is used to adjust the average metric of the new statesdetermined by (1/Jk_(n))Σd_(jn+1). If the number of retained states isexactly the target of 12, then (½) of the average metric is selected asthe target. Note that the average is used to provide an estimate of themedian. If the number of retained states is less than the target k_(n),the next threshold adjusts to be somewhat higher than ½ of the averagemetric. Alternatively, if the number of retained states is greater thanthe target, then the next threshold adjusts to be below (½) of theaverage metric. In that way, the threshold continually adapts to theactual number of retained states and fluctuates around the target K_(n).

A new threshold is typically determined for every new received symbol.All states are selected as retained states whose metric is less thanthreshold (t). This threshold approach has a complexity of the order(k_(n)) and is much more efficient for large k₁, k₂. It however does notguarantee that the resulting number of states selected from the newstates are in the range [k₁:k₂].

In case the number of states selected from the new states are not in therange of [k₁:k₂], the threshold has to be adjusted and selectionrepeated. If the threshold needs adjusting, it can be adjusted up ordown, as needed by a fixed percentage, e.g., 25% or some otherpercentage appropriate to the particular implementation. Thus, if thereare too few retained states, (<k₁), (t) can be adjusted up 25% andsimilarly, if there are too many retained states (>k₂), (t) can beadjusted down. The wider the range [k₁:k₂] the smaller the chance thatrepeat selection would be required. In simulation results for the caseswhere k₂=2k₁ repeat selection of the threshold is only required in about10% of iterations.

Other empirical approaches for determining the threshold t are alsopossible. For example, another approach is illustrated in FIG. 4. Mostof the operations illustrated in FIG. 4 are performed as part of thereduced state approach described herein and do not have to be repeatedfor determination of the threshold. In 401, the most likely state of theretained states is selected according to its state metric. That is, ofthe retained states, the most likely state is that having the lowestmetric, which is the output of the MLSE. Then, using the noisy estimatex_(n) of the received signal z_(n), the elements of the alphabet closestto x_(n) are selected and their transition metrics determined. Thoseelements will have metrics associated with them based on the Euclideandistance from the noisy estimate x_(n). In 407, the transition metric isselected. For the case of J=2, the transition metric selected is thesecond best transition metric. That provides a threshold for the nextestimate. Then in 407, the threshold is determined as the sum of theprevious best state metric and one of the transition metrics. The secondbest metric is chosen to provide a threshold that is high enough togenerate a sufficient number of retained states. If the threshold is settoo low or too higher, such that fewer than k₁ or greater than k₂ stateswould be retained, the threshold may be adjusted up or down accordingly.Note that if J is other than two, a metric other than second best may bechosen. This approach provides an adaptive threshold based on one of theretained states (the best retained state in the embodiment described).

The reduced state MLSE described herein may be used advantageously witha partial response class V equalizer, as illustrated, e.g., in FIG. 2 toprovide improved performance over channels with severe attenuation andspectral nulls. The combination of the MLSE and a PRV equalizer isdescribed as follows.

For each of k_(n) retained values, S_(jn), j=1, 2, . . . k_(n), of statevector S_(n), and received noisy signal z_(n), the MLSE calculates anoisy estimate of sent signal x_(jn):$x_{j\quad n} = {z_{n} - {\sum\limits_{i = 1}^{M - 1}\quad {x_{n - i}h_{i}}}}$

In the equation above, M is the memory of the channel in symbolintervals and z_(n) is the noisy output of the equalizer. For use withthe PRV equalizer, in order to determine an estimate of the sent signal,the MLSE has to subtract out the second difference of symbols spaced bytwo symbol intervals as previously described (−2x_(n−2)+x_(n−4))).Therefore in the equation above, h=−2 for i=2 and h=1 for i=4. BecauseM=5, four history terms have to be stored for each retained state. Onlytwo of the history terms are used during any particular generation ofthe noisy estimate. A noisy estimate is generated for each of theretained states using the stored history values and the output of thePRV.

The next operation of the MLSE is to determine J elements from alphabetA for each estimate x_(jn), such that the Euclidean distance from theseelements to x_(jn) is the smallest. Assuming J is two, the two alphabetelements closest to each noisy estimate, x_(jn) are selected. If thenoisy estimate is below the lowest alphabet element or above the largestelement, the first or last two alphabet element are chosen,respectively.

Finally, the MLSE calculates (J)(k_(n)) temporary new states for statevector S_(n+1), and state metric D_(n+1), having a metric associatedwith each of the states in state vector S_(n−1).

Thus, continuing with the assumption that k_(n)=8 and J=2, 16 new stateswill be calculated. In addition, 16 state metrics are determined thatare associated with the new states. The new state metrics are based onthe prior state metric and the transition metric associated with the Jtransitions selected. The transition metrics, as is known in the art,are determined according to the Euclidean distance between the Jalphabet points and the noisy estimate.

Each new state is generated using the elements (alphabet values) fromthe previous state, by deleting the oldest element and inserting as thenewest element, the alphabet element representing the received valuethat transitioned the previous state in S_(n) to the current state inS_(n−1).

As described previously, various threshold techniques may be employed toreduce the new states to the appropriate number. If necessary, thethreshold may be increased or decreased to increase or reduce,respectively, the number of retained states. As with the rest of theMLSE and the PRV equalizer described herein, the operation ofdetermining and adjusting the threshold may be implemented in softwareinstructions operating on a digital signal processor and/or a generalpurpose processor.

The description of the invention set forth herein is illustrative, andis not intended to limit the scope of the invention as set forth in thefollowing claims. For instance while the reduced state MLSE describedherein was described in association with a partial response class Vequalizer, the reduced state MLSE described herein may be employedwherever Viterbi decoding has been used, for example in decodingconvolutionally encoded data or in conjunction with different classes ofpartial response equalizers. Additionally, while the communicationdevice described was a modem complying with the V.90 standard, othertelecommunication devices may also benefit from the invention describedherein and the invention should not be considered as being limited to aparticular type of communication device or to a particular modemimplementation. Variations and modifications of the embodimentsdisclosed herein, may be made based on the description set forth herein,without departing from the scope and spirit of the invention as setforth in the following claims.

What is claimed is:
 1. A method of implementing a reduced state maximumlikelihood sequence estimator in a communications system to decode asignal, comprising: retaining k_(n) states of a total number of Kstates, k_(n)<K, each retained state having an associated state metric;determining (J)(k_(n)) new states based on k_(n) previous states and amost recently received sample, using J transitions, J being a less thanL, where L is a size of a symbol alphabet; determining (J)(k_(n)) newstate metrics respectively associated with each new state; performing acomparison of the new state metrics to a threshold; and retaining thosenew states whose metric does not exceed the threshold.
 2. The method asrecited in claim 1 wherein the threshold is adaptively determined. 3.The method as recited in claim 1 wherein the number of retained statesk_(n), varies between a lower bound and an upper bound.
 4. The method asrecited in claim 1 wherein the threshold is fixed.
 5. The method asrecited in claim 2 wherein the threshold is determined according to anaverage of the metrics of the new states and the number of retainedstates k_(n).
 6. The method as recited in claim 5 wherein the threshold(t) is determined according to t=((k₁+k₂)/(2Jk_(n)))(1/Jk_(n))Σd_(jn+1), where Σd_(jn+1) is the sum of the metricsof the new states and wherein k₁+k₂ are the lower and higher bound,respectively, of k_(n).
 7. The method as recited in claim 2 whereindetermining the threshold comprises: determining a most likely one ofthe retained states according to its associated metric; determine the Jtransitions from the most likely retained state and associated Jtransition metrics; determining the threshold to be the metricassociated with the most likely retained state added to a selectedtransition metric associated with one of the J transitions.
 8. Themethod as recited in claim 7 wherein the selected transition metric isnot the lowest of the J transition metrics.
 9. The method as recited inclaim 7 wherein the selected transition metric is second lowest of the Jtransitions.
 10. A modem comprising an equalizer having a partialresponse class V (PRV), the equalizer coupled in a receive path of themodem.
 11. The modem as recited in claim 10 wherein an alphabet size ofpossible values of a transmitted symbol is large, so as to makecomputational complexity of full state Viterbi decoding unmanageable.12. The modem as recited in claim 11 wherein the alphabet size >32. 13.The modem as recited in claim 10 further comprising a reduced statemaximum likelihood sequence estimator (MLSE) coupled to the PRVequalizer.
 14. A modem comprising: a partial response class V (PRV)equalizer coupled in a receive path of the modem; a reduced statemaximum likelihood sequence estimator (MLSE) coupled to the PRVequalizer; and wherein the MLSE further comprises: a plurality k_(n) ofretained states, each of the retained states having an associated statemetric, k_(n) being less than a total number of possible states; a setof (J)(k_(n)) new states, determined according to J transitions, J beinga less than a size of a symbol alphabet, and an associated set of newstate metrics; and a threshold for comparison to the new state metrics.15. The modem as recited in claim 14 wherein the threshold is adaptive.16. The modem as recited in claim 14 wherein k_(n) is variable.
 17. Themodem as recited in claim 14 wherein the MLSE, receiving the outputz_(n), of the PRV equalizer, calculates a noisy estimate of a sentsymbol, for each of the retained states.
 18. The modem as recited inclaim 17 wherein the noisy estimate x_(jn) of the sent signal isdetermined according to:$x_{j\quad n} = {z_{n} - {\sum\limits_{i = 1}^{M - 1}\quad {x_{n - i}h_{i}}}}$

wherein M−1 is the number of stored history terms and h=−2 for i=2 andh=1 for i=4.
 19. A modem comprising a reduced state maximum likelihoodsequence estimator (MLSE) in a receive path of the modem, wherein thereduced state maximum likelihood sequence estimator further includes anadaptive threshold for comparison to potential new state metrics, forlimiting a number of retained states, wherein the adaptive threshold isadaptive according to a previous number of previously retained states.20. The modem as recited in claim 19 further comprising a partialresponse equalizer operatively coupled to provide a partial response tothe reduced state maximum likelihood sequence estimator.
 21. The modemas recited in claim 20 wherein the partial response equalizer is aT-spaced equalizer.
 22. The modem as recited in claim 19 wherein thereduced state maximum likelihood sequence estimator retains a varyingnumber of reduced states.
 23. A modem comprising a reduced state maximumlikelihood sequence estimator (MLSE) in a receive path of the modemwherein the reduced state maximum likelihood sequence estimator furthermaintains storage encoding: a plurality k_(n) of retained states, eachof the retained states having an associated state metric, k_(n) beingless than a total number of possible states; a set of (J)(k_(n)) newstates, determined according to J transitions, J being a less than asize of a symbol alphabet, and an associated set of new state metrics;and a threshold for comparison to the new state metrics.
 24. A method ofoperating a modem comprising generating a partial response using anequalizer on a received signal, the equalizer having a partial responseclass V (PRV).
 25. The method as recited in claim 24 further comprisingproviding the output of the PRV equalizer to a reduced state maximumlikelihood sequence estimator.
 26. The method as recited in claim 25further comprising generating an estimate of the transmitted symbolusing a variable number of reduced states and the received signal.
 27. Amethod of operating a modem comprising: generating a partial responseusing an equalizer on a received signal, the equalizer having a partialresponse class V (PRV); providing the output of the PRV equalizer to areduced state maximum likelihood sequence estimator; and generating anestimate of the transmitted symbol using a variable number of reducedstates and the received signal, wherein the variable number of thereduced states is determined as a function of a metric of one of thereduced states and an associated transition metric.
 28. A method ofoperating a modem comprising: generating a partial response using anequalizer on a received signal, the equalizer having a partial responseclass V (PRV); providing the output of the PRV equalizer to a reducedstate maximum likelihood sequence estimator; and generating an estimateof the transmitted symbol using a variable number of reduced states andthe received signal, wherein the variable number of reduced states isdetermined according to an adaptively determined threshold.
 29. A methodof operating a modem comprising: providing a most likely estimate of atransmitted symbol using a reduced state maximum likelihood sequenceestimator; generating an estimate of the transmitted symbol using avariable number of reduced states in the maximum likelihood sequenceestimator; and determining the variable number of reduced statesaccording to an empirically determined metric threshold, wherein theempirically determined metric threshold is empirically determinedaccording to a previous number of previously retained states.
 30. Themethod as recited in claim 29 further comprising providing a partialresponse from a partial response equalizer to the reduced state maximumlikelihood sequence estimator.
 31. The method as recited in claim 29,wherein the threshold is determined according to a metric of a mostlikely state and a selected one of a plurality of associated transitionmetrics.
 32. The method as recited in 30 wherein the partial responseequalizer is a class V partial response equalizer.
 33. A method ofimplementing a reduced state maximum likelihood sequence detectorcomprising: retaining a varying number of states, the number of retainedstates being determined according to an adaptive threshold; anddetermining the number of retained states by comparing the metrics ofpotential retained states to the adaptive threshold, wherein theadaptive threshold is adaptive according to the previous number ofpreviously retained states.
 34. The method as recited in claim 33wherein the number of retained states k_(n) varies between upper andlower bounds, k₁≦k_(n)≦k₂.
 35. The method as recited in claim 33 whereinthe threshold is adjusted if the number of retained states is less thank₁ and if the number of retained states is greater than k₂.
 36. A methodof decoding a received signal, comprising: retaining a first pluralityof retained states, the first plurality being less than a number ofpossible states; dynamically determining a threshold for comparison withstate metrics; determining a second plurality of new states andassociated state metrics; and keeping as new retained states those newstates having a metric less than the dynamically determined threshold.37. A computer program product implementing a reduced state maximumlikelihood sequence estimator for a received signal encoded in a machinereadable medium and executable on a computer to: retain k_(n) states ofa total number of K states, k_(n)<K, each retained state having anassociated state metric; determine (J)(k_(n)) new states based on k_(n)previous states and a most recently received sample, using Jtransitions, J being a less than L, where L is a size of a symbolalphabet; determine (J)(k_(n)) new state metrics associated respectivelywith each new state; perform a comparison of the new state metrics to athreshold; and keep as new retained states those states whose metricdoes not exceed the threshold.
 38. The computer program product asrecited in claim 37 encoded by or transmitted in at least one computerreadable medium selected from the set of a disk, tape, or othermagnetic, optical or electronic storage medium and a network, wireline,wireless or other communications medium.
 39. A communication devicecomprising: a reduced state maximum likelihood sequence estimatorcoupled into a received path of the communication device; retained statestorage; and means for selecting new states to be retained in theretained state storage based on a dynamically calculated state metricthreshold, wherein the dynamically calculated state metric threshold isdynamically calculated according to a previous number of previouslyretained states.
 40. The method as recited in claim 36 wherein thethreshold is determined for every new received symbol.